Local strong convexity and local Lipschitz continuity of the gradient of convex functions

DSA: Decentralized Double Stochastic Averaging Gradient Algorithm

This paper considers convex optimization problems where nodes of a network have access to summands of a global objective. Each of these local objectives is further assumed to be an average of a finite set of functions. The motivation for this setup is to solve large scale machine learning problems where elements of the training set are distributed to multiple computational elements. The decentr...

Convergence Rates for Deterministic and Stochastic Subgradient Methods Without Lipschitz Continuity

We extend the classic convergence rate theory for subgradient methods to apply to non-Lipschitz functions. For the deterministic projected subgradient method, we present a global O(1/ √ T ) convergence rate for any convex function which is locally Lipschitz around its minimizers. This approach is based on Shor’s classic subgradient analysis and implies generalizations of the standard convergenc...

Convexity and Geodesic Metric Spaces

In this paper, we first present a preliminary study on metric segments and geodesics in metric spaces. Then we recall the concept of d-convexity of sets and functions in the sense of Menger and study some properties of d-convex sets and d-convex functions as well as extreme points and faces of d-convex sets in normed spaces. Finally we study the continuity of d-convex functions in geodesic metr...

Gradient methods for convex minimization: better rates under weaker conditions

The convergence behavior of gradient methods for minimizing convex differentiable functions is one of the core questions in convex optimization. This paper shows that their well-known complexities can be achieved under conditions weaker than the commonly accepted ones. We relax the common gradient Lipschitz-continuity condition and strong convexity condition to ones that hold only over certain ...

Approximate Convexity and Submonotonicity in Locally Convex Spaces